Video data need to be processed in many different situations and applications. A very common kind of processing of video data is encoding and decoding of video data, typically for the purpose of compressing the video data at the source/encoder side by video encoding, and decompressing the encoded video data at the destination/decoder side by video decoding.
High Efficiency Video Coding (HEVC), also referred to as H.265, is a video compression standard. HEVC is developed jointly by the ISO/IEC Moving Picture Experts Group (MPEG) and Telecommunication Standardization Sector (ITU-T) Video Coding Experts Group (VCEG) as ISO/IEC 23008-2 MPEG-H Part 2 and ITU-T H.265. MPEG and VCEG have established a Joint Collaborative Team on Video Coding (JCT-VC) to develop the HEVC standard.
In a video coding or compression system compliant with, for instance, the HEVC standard, the video data is subjected to various processing steps, including for instance prediction, residual, transformation, quantization, deblocking and adaptive loop filtering. Along the processing path in the video coding or compression system, certain characteristics of the video data may be altered from the original video data due to the operations in the processing steps which the video data is subjected to. For example, artefacts in the form of shifts in image intensity (e.g. chrominance or luminance) may occur for pixels in a video frame, and/or between successive video frames. Such artefacts may be visually noticeable; therefore measures may be taken in order to compensate for the artefacts in an attempt to remove or at least alleviate them.
In HEVC, an intensity compensation scheme known as Sample Adaptive Offset (SAO) is used. The SAO scheme classifies each pixel in the video data into one of multiple SAO categories according to a given context. The context may for instance be the pixel intensity of the video data, which is often referred to as “SAO band offsets”. Alternatively or additionally, the context may be a pixel value relation between the current pixel and its neighbouring pixels, which is often referred to as “SAO edge offsets”. In the latter case, the SAO categories represent typical edge artefacts and are associated with respective corresponding offset values to be applied to pixels in the respective SAO category so as to compensate for the edge artefact in question. Depending on where the adaptive offset is applied, the video data may represent reconstructed video data, video data which has undergone deblocking, adaptive loop-filtered video data, or other video data in an intermediate stage during the encoding or decoding process. Hence, SAO may be used in HEVC after the deblocking filter process if a deblocking filter is used, otherwise directly after reconstruction of prediction and residual. SAO modifies the picture that is to be displayed or stored in a reference picture buffer. The reference picture buffer keeps pictures for later use for inter prediction of other pictures. If adaptive loop filtering is used it is normally performed after SAO.
More specifically, SAO compensation in HEVC involves four SAO edge offset categories. The first category represents a case where the current pixel (or more specifically its intensity value) is at a local minimum compared to its neighbouring two pixels in a selected direction—horizontal (0 degrees), vertical (90 degrees), or diagonal (135 or 45 degrees). The second category represents a case where the current pixel is equal to one of its neighbours but lower than the other neighbour in the selected direction. The third category represents a case where the current pixel is equal to one of its neighbours but higher than the other neighbour in the selected direction. The fourth category represents a case where the current pixel is at a local maximum compared to its neighbouring two pixels in the selected direction.
These four SAO categories are shown in FIG. 2a and will be explained in more detail later on in this document.
Deblocking filters are used in the video coding standards in order to combat blocking artefacts. The blocking artefacts arise because the original video frames are split into blocks which are processed relatively independently. The blocking artefacts can, for instance, arise due to different intra predictions of the blocks, quantization effects and motion compensation.
In HEVC there is a deblocking filter, also denoted loop filter, after prediction and residual reconstruction, but before storage of the reconstruction for later reference when encoding or decoding the subsequent frames. The deblocking filtering consists of several steps such as filter decisions, filtering operations, a clipping function and change of pixel values.
Currently SAO band and edge offset in HEVC have four offsets per Largest Coding Unit (LCU). Encoding these offsets is an overhead that makes many LCUs avoiding use of SAO edge or band offsets. Coding several offsets also consumes memory. A problem is hence that the several offsets consumes too much memory.
Another problem is that it costs many bits to use band or edge offset with large magnitudes for SAO in HEVC. For low bit rates where offset with large magnitudes are more probable it cause unnecessary overhead or less usage of offsets with large magnitudes
A further problem with band offset is that it requires a pixel based table lookup to perform the band offset operation. The table has a size of 32 and need to be updated before processing each LCU.
Yet another problem with edge and band offset is that it requires clipping operations to allow a modification with band offset to stay within the bit depth of the video signal. For band offset in the HEVC reference software this is done by having a larger table 32+maxoffset/8-minoffset/8 where the ends of the table goes to minimum respectively maximum value according to the bitdepth. For edge offset this is done by having a table as large as the range of pixel values+maxoffset−minoffset. This is not mentioned in the HEVC working draft.